Safety signal identification for COVID-19 bivalent booster vaccination using tree-based scan statistics in the Vaccine Safety Datalink.

BACKGROUND: Traditional active vaccine safety monitoring involves pre-specifying health outcomes and biologically plausible outcome-specific time windows of concern, limiting the adverse events that can be evaluated. In this study, we used tree-based scan statistics to look broadly for >60,000 possible adverse events after bivalent COVID-19 vaccination. METHODS: Vaccine Safety Datalink enrollees aged ≥5 years receiving Moderna or Pfizer-BioNTech bivalent COVID-19 vaccine through November 2022 were followed for 56 days post-vaccination. Incident diagnoses in inpatient or emergency department settings were analyzed for clustering within the hierarchical ICD-10-CM diagnosis code "tree" and temporally within post-vaccination follow-up. The conditional self-controlled tree-temporal scan statistic was used, conditioning on total number of cases of each diagnosis and total number of cases of any diagnosis occurring during the scanning risk window across the entire tree. P = 0.01 was the pre-specified cut-off for statistical significance. RESULTS: Analysis included 352,509 doses of Moderna and 979,189 doses of Pfizer-BioNTech bivalent vaccines. After Moderna vaccination, no statistically significant clusters were found. After Pfizer-BioNTech, there were clusters of unspecified adverse events (Days 1-3, p = 0.0001-0.0007), influenza (Days 35-56, p = 0.0001), cough (Days 44-55, p = 0.0002), and COVID-19 (Days 52-56, p = 0.0004). CONCLUSIONS: For Pfizer-BioNTech only, we detected clusters of: (1) unspecified adverse effects, as have been observed in other vaccine studies using this method, and (2) respiratory disease toward the end of follow-up. The respiratory clusters were likely due to overlap of follow-up with the spread of respiratory syncytial virus, influenza, and COVID-19, i.e., confounding by seasonality. The untargeted nature of the method and its inherent adjustment for the many diagnoses and risk intervals evaluated are unique advantages. Limitations include susceptibility to time-varying confounding, lower statistical power for assessing risks of specific outcomes than in traditional studies targeting fewer outcomes, and the possibility of missing adverse events not strongly clustered in time or within the "tree."

Tree-based data mining for safety assessment of first COVID-19 booster doses in the Vaccine Safety Datalink.

BACKGROUND: The Centers for Disease Control and Prevention's Vaccine Safety Datalink (VSD) has been performing safety surveillance for COVID-19 vaccines since their earliest authorization in the United States. Complementing its real-time surveillance for pre-specified health outcomes using pre-specified risk intervals, the VSD conducts tree-based data-mining to look for clustering of a broad range of health outcomes after COVID-19 vaccination. This study's objective was to use this untargeted, hypothesis-generating approach to assess the safety of first booster doses of Pfizer-BioNTech (BNT162b2), Moderna (mRNA-1273), and Janssen (Ad26.COV2.S) COVID-19 vaccines. METHODS: VSD enrollees receiving a first booster of COVID-19 vaccine through April 2, 2022 were followed for 56 days. Incident diagnoses in inpatient or emergency department settings were analyzed for clustering within both the hierarchical ICD-10-CM code structure and the follow-up period. The self-controlled tree-temporal scan statistic was used, conditioning on the total number of cases for each diagnosis. P-values were estimated by Monte Carlo simulation; p = 0.01 was pre-specified as the cut-off for statistical significance of clusters. RESULTS: More than 2.4 and 1.8 million subjects received Pfizer-BioNTech and Moderna boosters after an mRNA primary series, respectively. Clusters of urticaria/allergy/rash were found during Days 10-15 after the Moderna booster (p = 0.0001). Other outcomes that clustered after mRNA boosters, mostly with p = 0.0001, included unspecified adverse effects, common vaccine-associated reactions like fever and myalgia, and COVID-19. COVID-19 clusters were in Days 1-10 after booster receipt, before boosters would have become effective. There were no noteworthy clusters after boosters following primary Janssen vaccination. CONCLUSIONS: In this untargeted data-mining study of COVID-19 booster vaccination, a cluster of delayed-onset urticaria/allergy/rash was detected after the Moderna booster, as has been reported after Moderna vaccination previously. Other clusters after mRNA boosters were of unspecified or common adverse effects and COVID-19, the latter evidently reflecting immunity to COVID-19 after 10 days.

Risk of myocarditis and pericarditis following BNT162b2 and mRNA-1273 COVID-19 vaccination.

BACKGROUND: Evidence indicates that mRNA COVID-19 vaccination is associated with risk of myocarditis and possibly pericarditis, especially in young males. It is not clear if risk differs between mRNA-1273 versus BNT162b2. We assessed if risk differs using comprehensive health records on a diverse population. METHODS: Members 18-39 years of age at eight integrated healthcare-delivery systems were monitored using data updated weekly and supplemented with medical record review of myocarditis and pericarditis cases. Incidence of myocarditis and pericarditis events that occurred among vaccine recipients 0 to 7 days after either dose 1 or 2 of a messenger RNA (mRNA) vaccine was compared with that of vaccinated concurrent comparators who, on the same calendar day, had received their most recent dose 22 to 42 days earlier. Rate ratios (RRs) were estimated by conditional Poisson regression, adjusted for age, sex, race and ethnicity, health plan, and calendar day. Head-to-head comparison directly assessed risk following mRNA-1273 versus BNT162b2 during 0-7 days post-vaccination. RESULTS: From December 14, 2020 - January 15, 2022 there were 41 cases after 2,891,498 doses of BNT162b2 and 38 cases after 1,803,267 doses of mRNA-1273. Cases had similar demographic and clinical characteristics. Most were hospitalized for ≤1 day; none required intensive care. During days 0-7 after dose 2 of BNT162b2, the incidence was 14.3 (CI: 6.5-34.9) times higher than the comparison interval, amounting to 22.4 excess cases per million doses; after mRNA-1273 the incidence was 18.8 (CI: 6.7-64.9) times higher than the comparison interval, amounting to 31.2 excess cases per million doses. In head-to-head comparisons 0-7 days after either dose, risk was moderately higher after mRNA-1273 than after BNT162b2 (RR: 1.61, CI 1.02-2.54). CONCLUSIONS: Both vaccines were associated with increased risk of myocarditis and pericarditis in 18-39-year-olds. Risk estimates were modestly higher after mRNA-1273 than after BNT162b2.

Multivariate scan statistics for disease surveillance.

In disease surveillance, there are often many different data sets or data groupings for which we wish to do surveillance. If each data set is analysed separately rather than combined, the statistical power to detect an outbreak that is present in all data sets may suffer due to low numbers in each. On the other hand, if the data sets are added by taking the sum of the counts, then a signal that is primarily present in one data set may be hidden due to random noise in the other data sets. In this paper, we present an extension of the spatial and space-time scan statistic that simultaneously incorporates multiple data sets into a single likelihood function, so that a signal is generated whether it occurs in only one or in multiple data sets. This is done by defining the combined log likelihood as the sum of the individual log likelihoods for those data sets for which the observed case count is more than the expected. We also present another extension, where the concept of combining likelihoods from different data sets is used to adjust for covariates. Using data from the National Bioterrorism Syndromic Surveillance Demonstration Project, we illustrate the new method using physician telephone calls, regular physician visits and urgent care visits by Harvard Pilgrim Health Care members cared for by Harvard Vanguard Medical Associates, a large multi-specialty group practice in Massachusetts. For upper and lower gastrointestinal (GI) illness, there were on average 20 telephone calls, nine urgent care visits and 22 regular physician visits per day. The strongest signal was generated by a single data set and due to a familial outbreak of pinworm disease. The second and third strongest signals were generated by the combined strength of two of the three data sets.

Evaluating spatial surveillance: detection of known outbreaks in real data.

Since the anthrax attacks of October 2001 and the SARS outbreaks of recent years, there has been an increasing interest in developing surveillance systems to aid in the early detection of such illness. Systems have been established which do this is by monitoring primary health-care visits, pharmacy sales, absenteeism records, and other non-traditional sources of data. While many resources have been invested in establishing such systems, relatively little effort has as yet been expended in evaluating their performance. One way to evaluate a given surveillance system is to compare the signals it generates with known outbreaks identified in other systems. In public health practice, for example, public health departments investigate reports of illness and sometimes track hospital admissions. Comparison of new systems with extant systems cannot generate estimates of test characteristics such as sensitivity and specificity, since the actual number of positives and negatives cannot be known. However, the comparison can reveal whether a new or proposed system's signals match outbreaks detected by the existing system. This could help support or reject the new system as an alternative or complement to the extant system. We propose three methods to test the null hypothesis that the new system does not signal true outbreaks more often than would be expected by chance. The methods differ in the restrictiveness of the assumptions required. Each test may detect weaknesses in the new system, depending on the distribution of outbreaks and can be used to construct confidence limits on the agreement between the new system's signals and the outbreaks, given the distribution of the signals. They can be used to assess whether the new system works in that it detects the outbreaks better than chance would suggest and can also determine if the new systems' signals are generated earlier than an extant system.